Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing flaky tests #4267

Merged
merged 1 commit into from
Apr 16, 2024
Merged

Fixing flaky tests #4267

merged 1 commit into from
Apr 16, 2024

Conversation

JanuszJakubiec
Copy link
Contributor

@JanuszJakubiec JanuszJakubiec commented Apr 16, 2024

Adding a function to assert the connection result. Allowing to return {error, connection_closed} in tests where the error was expected.

The proposed solution, of running the tests sequentially didn't solve the problem. Therefore it was proposed to add the error {error, connection_closed} as an acceptable result of tests where the SSL certificate wasn't valid.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Apr 16, 2024

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / d626352
Reports root/ big
OK: 437 / Failed: 0 / User-skipped: 43 / Auto-skipped: 0


small_tests_25 / small_tests / d626352
Reports root / small


small_tests_26 / small_tests / d626352
Reports root / small


small_tests_26_arm64 / small_tests / d626352
Reports root / small


ldap_mnesia_25 / ldap_mnesia / d626352
Reports root/ big
OK: 3174 / Failed: 1 / User-skipped: 897 / Auto-skipped: 0

graphql_roster_SUITE:user_roster:user_decline_subscription_ask
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"bob_user_decline_subscription_ask_1546@localhost/res1">>,
          escalus_tcp,<0.5093.1>,
          [{event_manager,<0.5091.1>},
           {server,<<"localhost">>},
           {username,<<"bOb_user_decline_subscription_ask_1546">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.5091.1>},
            {server,<<"localhost">>},
            {username,
              <<"bOb_user_decline_subscription_ask_1546">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"bob_user_decline_subscription_ask_1546">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"bOb_user_decline_subscription_ask_1546">>},
           {server,<<"localhost">>},
           {password,<<"makrolika">>},
           {stream_id,<<"b75dbfd0a85f046c">>}]},
        1],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {graphql_roster_SUITE,user_decline_subscription_ask_story,3,
       [{file,
          "/home/circleci/project/big_tests/tests/graphql_roster_SUITE.erl"},
        {line,524}]},
     {escalus_story,story,4,
       [{file,
          "/home/circleci/project/big_tests/_build/d...

Report log


ldap_mnesia_26 / ldap_mnesia / d626352
Reports root/ big
OK: 3175 / Failed: 0 / User-skipped: 897 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / d626352
Reports root/ big
OK: 5399 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / d626352
Reports root/ big
OK: 5426 / Failed: 1 / User-skipped: 111 / Auto-skipped: 5

bosh_SUITE:essential_https:accept_higher_hold_value
{error,
  {{assertEqual,
     [{module,bosh_SUITE},
      {line,261},
      {expression,"get_bosh_sessions ( )"},
      {expected,[]},
      {value,
        [{bosh_session,<<"61fb001a90ba6d9ba87cb391661673272fd80f90">>,
           <9328.10261.0>}]}]},
   [{bosh_SUITE,accept_higher_hold_value,1,
      [{file,"/home/circleci/project/big_tests/tests/bosh_SUITE.erl"},
       {line,261}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / d626352
Reports root/ big
OK: 5432 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / d626352
Reports root/ big
OK: 5347 / Failed: 0 / User-skipped: 178 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / d626352
Reports root/ big
OK: 5429 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / d626352
Reports root/ big
OK: 5821 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / d626352
Reports root/ big
OK: 3315 / Failed: 0 / User-skipped: 757 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / d626352
Reports root/ big
OK: 5821 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / d626352
Reports root/ big
OK: 5800 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / d626352
Reports root/ big
OK: 5818 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0

Copy link

codecov bot commented Apr 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.43%. Comparing base (4c125a0) to head (915cbd3).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4267   +/-   ##
=======================================
  Coverage   84.42%   84.43%           
=======================================
  Files         552      552           
  Lines       33554    33554           
=======================================
+ Hits        28328    28330    +2     
+ Misses       5226     5224    -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JanuszJakubiec JanuszJakubiec marked this pull request as ready for review April 16, 2024 10:52
@mongoose-im
Copy link
Collaborator

mongoose-im commented Apr 16, 2024

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 22cf3aa
Reports root/ big
OK: 437 / Failed: 0 / User-skipped: 43 / Auto-skipped: 0


small_tests_25 / small_tests / 22cf3aa
Reports root / small


small_tests_26_arm64 / small_tests / 22cf3aa
Reports root / small


small_tests_26 / small_tests / 22cf3aa
Reports root / small


ldap_mnesia_25 / ldap_mnesia / 22cf3aa
Reports root/ big
OK: 2284 / Failed: 0 / User-skipped: 897 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 22cf3aa
Reports root/ big
OK: 2284 / Failed: 0 / User-skipped: 897 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 22cf3aa
Reports root/ big
OK: 4508 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 22cf3aa
Reports root/ big
OK: 4541 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 22cf3aa
Reports root/ big
OK: 4456 / Failed: 0 / User-skipped: 178 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / 22cf3aa
Reports root/ big
OK: 2424 / Failed: 0 / User-skipped: 757 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 22cf3aa
Reports root/ big
OK: 4538 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 22cf3aa
Reports root/ big
OK: 4930 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 22cf3aa
Reports root/ big
OK: 4930 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 22cf3aa
Reports root/ big
OK: 4909 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 22cf3aa
Reports root/ big
OK: 4541 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 22cf3aa
Reports root/ big
OK: 4927 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0

Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good as a temporary solution 👍
How about squashing the two commits together?

@mongoose-im
Copy link
Collaborator

mongoose-im commented Apr 16, 2024

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 915cbd3
Reports root/ big
OK: 437 / Failed: 0 / User-skipped: 43 / Auto-skipped: 0


small_tests_25 / small_tests / 915cbd3
Reports root / small


small_tests_26 / small_tests / 915cbd3
Reports root / small


small_tests_26_arm64 / small_tests / 915cbd3
Reports root / small


internal_mnesia_26 / internal_mnesia / 915cbd3
Reports root/ big
OK: 330 / Failed: 9 / User-skipped: 0 / Auto-skipped: 2705

cluster_commands_SUITE:join_twice_using_rpc
{error,
  {{badrpc,timeout},
   [{distributed_helper,rpc,
      [#{timeout => 60000,node => mongooseim2@localhost},
       mongoose_cluster,join,
       [mongooseim@localhost]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {cluster_commands_SUITE,join_twice_using_rpc,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,285}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

cluster_commands_SUITE:join_twice_in_parallel_using_rpc
{'EXIT',{{badrpc,timeout},
     {distributed_helper,rpc,
               [#{timeout => 60000,
                node => mongooseim2@localhost},
                mongoose_cluster,join,
                [mongooseim@localhost]],
               [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                {line,140}]}}}

Report log

cluster_commands_SUITE:cluster_of_three
{error,
  {{assertEqual,
     [{module,cluster_commands_SUITE},
      {line,346},
      {expression,"OpCode1"},
      {expected,0},
      {value,1}]},
   [{cluster_commands_SUITE,cluster_of_three,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,346}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

cluster_commands_SUITE:leave_the_three
{error,
  {{badrpc,timeout},
   [{distributed_helper,rpc,
      [#{timeout => 60000,node => mongooseim2@localhost},
       mongoose_cluster,join,
       [mongooseim@localhost]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {cluster_commands_SUITE,leave_the_three,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,358}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

cluster_commands_SUITE:remove_alive_from_cluster
{error,
  {{badrpc,timeout},
   [{distributed_helper,rpc,
      [#{timeout => 60000,node => mongooseim2@localhost},
       mongoose_cluster,join,
       [mongooseim@localhost]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {cluster_commands_SUITE,remove_alive_from_cluster,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,400}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

component_SUITE:subdomain:init_per_group
{'EXIT',{{badrpc,timeout},
     [{distributed_helper,rpc,
                [#{timeout => 30000,
                 node => mongooseim@localhost},
                 application,start,
                 [mongooseim]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {ejabberd_node_utils,restart_application,2,
                 [{file,"/home/circleci/project/big_tests/tests/ejabberd_node_utils.erl"},
                {line,77}]},
      {component_SUITE,add_domain,1,
               [{file,"/home/circleci/project/big_tests/tests/component_SUITE.erl"},
              {line,548}]},
      {component_SUITE,init_per_group,2,
               [{file,"/home/circleci/project/big_tests/tests/component_SUITE.erl"},
              {line,93}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

component_SUITE:hidden_components:init_per_group
{'EXIT',
  {{badrpc,
     {'EXIT',
       {badarg,
         [{ets,lookup,
            [mongoose_domain_core,<<"localhost">>],
            [{error_info,
               #{cause => id,module => erl_stdlib_errors}}]},
          {mongoose_domain_core,get_host_type,1,
            [{file,
               "/home/circleci/project/src/domain/mongoose_domain_core.erl"},
             {line,48}]},
          {ejabberd_auth,call_auth_modules_for_domain,3,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,512}]},
          {ejabberd_auth,does_user_exist,1,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,310}]},
          {ejabberd_auth,try_register,2,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,220}]},
          {mongoose_account_api,register_user,3,
            [{file,
               "/home/circleci/project/src/mongoose_account_api.erl"},
             {line,78}]},
          {ejabberd_admin,register,3,[]}]}}},
   [{escalus_rpc,call_with_cookie_match,
      [mongooseim@localhost,ejabberd_admin,register,
       [<<"alicE">>,<<"localhost">>,<<"matygrysa">>],
       3000,mongooseim],
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_rpc.erl"},
       {line,34}]},
    {lists,foreach_1,2,[{file,"lists.erl"},{line,1686}]},
    {escalus_ejabberd,create_users,2,
      [{file,
 ...

Report log

component_SUITE:distributed:init_per_group
{'EXIT',{{badrpc,timeout},
     [{distributed_helper,rpc,
                [#{timeout => 30000,
                 node => mongooseim2@localhost},
                 mongoose_cluster,join,
                 [mongooseim@localhost]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {distributed_helper,add_node_to_mnesia_cluster,2,
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,44}]},
      {distributed_helper,add_node_to_cluster,2,
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,36}]},
      {component_SUITE,init_per_group,2,
               [{file,"/home/circleci/project/big_tests/tests/component_SUITE.erl"},
              {line,100}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

connect_SUITE:init_per_suite
{'EXIT',
  {{badrpc,
     {'EXIT',
       {badarg,
         [{ets,lookup,
            [mongoose_domain_core,<<"localhost">>],
            [{error_info,
               #{cause => id,module => erl_stdlib_errors}}]},
          {mongoose_domain_core,get_host_type,1,
            [{file,
               "/home/circleci/project/src/domain/mongoose_domain_core.erl"},
             {line,48}]},
          {ejabberd_auth,call_auth_modules_for_domain,3,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,512}]},
          {ejabberd_auth,does_user_exist,1,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,310}]},
          {ejabberd_auth,try_register,2,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,220}]},
          {mongoose_account_api,register_user,3,
            [{file,
               "/home/circleci/project/src/mongoose_account_api.erl"},
             {line,78}]},
          {ejabberd_admin,register,3,[]}]}}},
   [{escalus_rpc,call_with_cookie_match,
      [mongooseim@localhost,ejabberd_admin,register,
       [<<"secure_joe">>,<<"localhost">>,<<"break_me">>],
       3000,mongooseim],
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_rpc.erl"},
       {line,34}]},
    {lists,foreach_1,2,[{file,"lists.erl"},{line,1686}]},
    {escalus_ejabberd,create_users,2,
      [{fil...

Report log

disco_and_caps_SUITE:disco_with_caps:init_per_group
{'EXIT',
  {{badrpc,
     {'EXIT',
       {badarg,
         [{ets,lookup,
            [mongoose_wpool,{rdbms,<<"localhost">>,default}],
            [{error_info,
               #{cause => id,module => erl_stdlib_errors}}]},
          {mongoose_wpool,get_pool,3,
            [{file,
               "/home/circleci/project/src/wpool/mongoose_wpool.erl"},
             {line,388}]},
          {mongoose_wpool,call,4,
            [{file,
               "/home/circleci/project/src/wpool/mongoose_wpool.erl"},
             {line,246}]},
          {mongoose_rdbms,sql_transaction,2,[]}]}}},
   [{distributed_helper,rpc,
      [#{node => mongooseim@localhost},
       mongoose_rdbms,sql_transaction,
       [<<"localhost">>,fun erlang:yield/0]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {mongoose_helper,is_rdbms_enabled,1,
      [{file,
         "/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
       {line,61}]},
    {mongoose_helper,get_backend_mnesia_rdbms,1,
      [{file,
         "/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
       {line,75}]},
    {config_parser_helper,mod_config_with_auto_backend,2,
      [{file,
         "/home/circleci/project/big_tests/../test/common/config_parser_helper.erl"},
       {line,852}]},
    {disco_and_caps_SUITE,required_modules,1,
      [{file,
         "/home/circleci/project/big_tests/tests/disco_and_caps_SUITE.erl"},
      ...

Report log

disco_and_caps_SUITE:disco_with_caps_and_extra_features:init_per_group
{'EXIT',
  {{badrpc,
     {'EXIT',
       {badarg,
         [{ets,lookup,
            [mongoose_wpool,{rdbms,<<"localhost">>,default}],
            [{error_info,
               #{cause => id,module => erl_stdlib_errors}}]},
          {mongoose_wpool,get_pool,3,
            [{file,
               "/home/circleci/project/src/wpool/mongoose_wpool.erl"},
             {line,388}]},
          {mongoose_wpool,call,4,
            [{file,
               "/home/circleci/project/src/wpool/mongoose_wpool.erl"},
             {line,246}]},
          {mongoose_rdbms,sql_transaction,2,[]}]}}},
   [{distributed_helper,rpc,
      [#{node => mongooseim@localhost},
       mongoose_rdbms,sql_transaction,
       [<<"localhost">>,fun erlang:yield/0]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {mongoose_helper,is_rdbms_enabled,1,
      [{file,
         "/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
       {line,61}]},
    {mongoose_helper,get_backend_mnesia_rdbms,1,
      [{file,
         "/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
       {line,75}]},
    {config_parser_helper,mod_config_with_auto_backend,2,
      [{file,
         "/home/circleci/project/big_tests/../test/common/config_parser_helper.erl"},
       {line,852}]},
    {disco_and_caps_SUITE,required_modules,1,
      [{file,
         "/home/circleci/project/big_tests/tests/disco_and_caps_SUITE.erl"},
      ...

Report log

disco_and_caps_SUITE:disco_with_extra_features:init_per_group
{'EXIT',
  {{badrpc,
     {'EXIT',
       {noproc,
         {gen_server,call,
           [gen_hook,
            {delete_handler,
              {disco_local_features,<<"localhost">>},
              {hook_handler,100,
                fun mod_disco:disco_local_features/3,
                #{host_type => <<"localhost">>,
                hook_name => disco_local_features,
                hook_tag => <<"localhost">>}}}]}}}},
   [{distributed_helper,rpc,
      [#{node => mongooseim@localhost},
       mongoose_modules,replace_modules,
       [<<"localhost">>,[],
      #{mod_disco =>
          #{iqdisc => one_queue,users_can_see_hidden_services => true,
          server_info =>
            [#{name => <<"abuse-addresses">>,
               urls => [<<"[email protected]">>]},
             #{name => <<"admin-addresses">>,
               modules => [mod_disco],
               urls =>
                 [<<"[email protected]">>,
                <<"[email protected]">>]},
             #{name => <<"sales-addresses">>,
               modules => [mod_pubsub],
               urls => [<<"[email protected]">>]}],
          extra_domains => [<<"eXtra.example.com">>]}}]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {disco_and_caps_SUITE,init_per_group,2,
      [{file,
         "/home/circleci/project/big_tests/tests/disco_and_caps_SUITE.erl"},
       {line,48}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,...

Report log

extdisco_SUITE:extdisco_not_configured:external_services_discovery_not_supported
{error,
  {{badrpc,
     {'EXIT',
       {badarg,
         [{ets,lookup,
            [mongoose_domain_core,<<"localhost">>],
            [{error_info,
               #{cause => id,module => erl_stdlib_errors}}]},
          {mongoose_domain_core,get_host_type,1,
            [{file,
               "/home/circleci/project/src/domain/mongoose_domain_core.erl"},
             {line,48}]},
          {ejabberd_auth,call_auth_modules_for_domain,3,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,512}]},
          {ejabberd_auth,does_user_exist,1,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,310}]},
          {ejabberd_auth,try_register,2,
            [{file,
               "/home/circleci/project/src/auth/ejabberd_auth.erl"},
             {line,220}]},
          {mongoose_account_api,register_user,3,
            [{file,
               "/home/circleci/project/src/mongoose_account_api.erl"},
             {line,78}]},
          {ejabberd_admin,register,3,[]}]}}},
   [{escalus_rpc,call_with_cookie_match,
      [mongooseim@localhost,ejabberd_admin,register,
       [<<"alicE_external_services_discovery_not_supported_507">>,
      <<"localhost">>,<<"matygrysa">>],
       3000,mongooseim],
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_rpc.erl"},
       {line,34}]},
    {lists,foreach_1,2,[{file,"lists.erl"},{line,1686}]},
 ...

Report log

extdisco_SUITE:extdisco_configured:init_per_group
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 mongoose_modules,replace_modules,
                 [<<"localhost">>,[],
                #{mod_extdisco =>
                    #{service =>
                      [#{port => 3478,type => stun,
                         host => <<"1.1.1.1">>,
                         password => <<"secret">>,
                         username => <<"username">>,
                         transport => <<"udp">>}],
                    iqdisc => no_queue}}]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {extdisco_SUITE,set_external_services,2,
              [{file,"/home/circleci/project/big_tests/tests/extdisco_SUITE.erl"},
               {line,317}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

extdisco_SUITE:multiple_extdisco_configured:init_per_group
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 mongoose_modules,replace_modules,
                 [<<"localhost">>,[],
                #{mod_extdisco =>
                    #{service =>
                      [#{port => 3478,type => stun,
                         host => <<"1.1.1.1">>,
                         password => <<"secret">>,
                         username => <<"username">>,
                         transport => <<"udp">>},
                       #{port => 3478,type => stun,
                         host => <<"1.1.1.1">>,
                         password => <<"secret">>,
                         username => <<"username">>,
                         transport => <<"udp">>},
                       #{port => 3478,type => turn,
                         host => <<"2.2.2.2">>,
                         password => <<"secret">>,
                         username => <<"username">>,
                         transport => <<"tcp">>}],
                    iqdisc => no_queue}}]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {extdisco_SUITE,set_external_services,2,
              [{file,"/home/circleci/project/big_tests/tests/extdisco_SUITE.erl"},
               {line,317}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_serv...

Report log

extdisco_SUITE:extdisco_required_elements_configured:init_per_group
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 mongoose_modules,replace_modules,
                 [<<"localhost">>,[],
                #{mod_extdisco =>
                    #{service =>
                      [#{type => ftp,
                         host => <<"3.3.3.3">>}],
                    iqdisc => no_queue}}]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {extdisco_SUITE,set_external_services,2,
              [{file,"/home/circleci/project/big_tests/tests/extdisco_SUITE.erl"},
               {line,317}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

gdpr_SUITE:init_per_suite
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 file,get_cwd,[]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {gdpr_SUITE,get_mim_cwd,0,
            [{file,"/home/circleci/project/big_tests/tests/gdpr_SUITE.erl"},
             {line,1725}]},
      {gdpr_SUITE,init_per_suite,1,
            [{file,"/home/circleci/project/big_tests/tests/gdpr_SUITE.erl"},
             {line,177}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_SUITE:init_per_suite
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 gen_mod,loaded_modules_with_opts,
                 [<<"localhost">>]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {dynamic_modules,'-save_modules/3-fun-0-',4,
               [{file,"/home/circleci/project/big_tests/tests/dynamic_modules.erl"},
              {line,16}]},
      {lists,foldl,3,[{file,"lists.erl"},{line,1594}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_sse_SUITE:admin:init_per_group
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 mongoose_config,get_opt,
                 [listen]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {graphql_helper,get_listener_config,2,
              [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
               {line,143}]},
      {graphql_helper,get_listener_opts,1,
              [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
               {line,198}]},
      {graphql_helper,init_admin_handler,1,
              [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
               {line,150}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_sse_SUITE:user:init_per_group
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 mongoose_graphql_commands,build_specs,
                 [user]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {graphql_helper,add_specs,1,
              [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
               {line,181}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_sse_SUITE:timeout:init_per_group
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 mongoose_config,get_opt,
                 [listen]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {mongoose_helper,get_listeners,2,
               [{file,"/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
              {line,494}]},
      {graphql_sse_SUITE,get_graphql_user_listener,0,
               [{file,"/home/circleci/project/big_tests/tests/graphql_sse_SUITE.erl"},
                {line,156}]},
      {graphql_sse_SUITE,init_per_group,2,
               [{file,"/home/circleci/project/big_tests/tests/graphql_sse_SUITE.erl"},
                {line,40}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_account_SUITE:init_per_suite
{'EXIT',
  {{badrpc,nodedown},
   [{distributed_helper,rpc,
      [#{node => mongooseim@localhost},mongoose_config,get_opt,[hosts]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {mongoose_helper,auth_modules,0,
      [{file,
         "/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
       {line,85}]},
    {graphql_account_SUITE,init_per_suite,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_account_SUITE.erl"},
       {line,93}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1390}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_domain_SUITE:init_per_suite
{'EXIT',
  {{badrpc,nodedown},
   [{distributed_helper,rpc,
      [#{node => mongooseim@localhost},
       mongoose_rdbms,sql_transaction,
       [<<"dummy auth">>,fun erlang:yield/0]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {mongoose_helper,is_rdbms_enabled,1,
      [{file,
         "/home/circleci/project/big_tests/../test/common/mongoose_helper.erl"},
       {line,61}]},
    {graphql_domain_SUITE,init_per_suite,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_domain_SUITE.erl"},
       {line,70}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1390}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

ct_framework:error_in_suite
{error,'graphql_inbox_SUITE:all/0 failed'}

Report log

graphql_last_SUITE:init_per_suite
{'EXIT',{{badrpc,nodedown},
     [{distributed_helper,rpc,
                [#{node => mongooseim@localhost},
                 gen_mod,loaded_modules_with_opts,
                 [<<"localhost">>]],
                [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                 {line,140}]},
      {dynamic_modules,'-save_modules/3-fun-0-',4,
               [{file,"/home/circleci/project/big_tests/tests/dynamic_modules.erl"},
              {line,16}]},
      {lists,foldl,3,[{file,"lists.erl"},{line,1594}]},
      {graphql_last_SUITE,init_per_suite,1,
                [{file,"/home/circleci/project/big_tests/tests/graphql_last_SUITE.erl"},
                 {line,137}]},
      {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
      {test_server,run_test_case_eval1,6,
             [{file,"test_server.erl"},{line,1390}]},
      {test_server,run_test_case_eval,9,
             [{file,"test_server.erl"},{line,1234}]}]}}

Report log

136 errors were truncated


ldap_mnesia_25 / ldap_mnesia / 915cbd3
Reports root/ big
OK: 2284 / Failed: 0 / User-skipped: 897 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 915cbd3
Reports root/ big
OK: 4541 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 915cbd3
Reports root/ big
OK: 2284 / Failed: 0 / User-skipped: 897 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 915cbd3
Reports root/ big
OK: 4508 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 915cbd3
Reports root/ big
OK: 4541 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 915cbd3
Reports root/ big
OK: 4538 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 915cbd3
Reports root/ big
OK: 4456 / Failed: 0 / User-skipped: 178 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 915cbd3
Reports root/ big
OK: 4930 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 915cbd3
Reports root/ big
OK: 4909 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 915cbd3
Reports root/ big
OK: 4929 / Failed: 1 / User-skipped: 118 / Auto-skipped: 0

sm_SUITE:ping_timeout
{error,{noproc,{gen_server,call,[<0.96334.0>,get_sm_h]}}}

Report log


mssql_mnesia_26 / odbc_mssql_mnesia / 915cbd3
Reports root/ big
OK: 4923 / Failed: 4 / User-skipped: 121 / Auto-skipped: 0

mod_blocking_SUITE:manage:add_another_user_to_blocklist
{error,{{assertion_failed,assert_many,false,
              [is_iq_result,#Fun<mod_blocking_SUITE.32.129816315>],
              [],[]},
    [{escalus_new_assert,assert_true,2,
               [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
                {line,84}]},
     {mod_blocking_SUITE,'-add_another_user_to_blocklist/1-fun-0-',2,
               [{file,"/home/circleci/project/big_tests/tests/mod_blocking_SUITE.erl"},
                {line,176}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1234}]}]}}

Report log

mod_blocking_SUITE:manage:remove_many_user_from_blocklist
{error,
  {{badmatch,
     {error,
       {connection_step_failed,
         {{escalus_session,session},
          {client,
            <<"bob_unnamed_2622_remove_many_user_from_blocklist_2630@localhost/res1">>,
            escalus_tcp,<0.64119.0>,undefined,
            [{event_client,
               [{event_manager,<0.64088.0>},
              {server,<<"localhost">>},
              {username,
                <<"bOb_unnamed_2622_remove_many_user_from_blocklist_2630">>},
              {resource,<<"res1">>}]},
             {resource,<<"res1">>},
             {username,
               <<"bob_unnamed_2622_remove_many_user_from_blocklist_2630">>},
             {server,<<"localhost">>},
             {host,<<"localhost">>},
             {port,5222},
             {auth,{escalus_auth,auth_plain}},
             {wspath,undefined},
             {username,
               <<"bOb_unnamed_2622_remove_many_user_from_blocklist_2630">>},
             {server,<<"localhost">>},
             {password,<<"makrolika">>},
             {stream_id,<<"fb7ad7f0c2f3683e">>}]},
          [{compression,false},
           {starttls,true},
           {stream_management,true},
           {advanced_message_processing,true},
           {client_state_indication,false},
           {sasl_mechanisms,[<<"SCRAM-SHA-256">>,<<"PLAIN">>]},
           {caps,undefined}]},
         {timeout,session_reply}}}},
   [{escalus_story,'-start_ready_clients/2-fun-0-',3,
      [{file,
         "/home/circleci/project/big_tests/_buil...

Report log

mod_blocking_SUITE:manage:remove_user_from_blocklist
{error,{{assertion_failed,assert_many,false,
              [is_iq_result,#Fun<mod_blocking_SUITE.32.129816315>],
              [],[]},
    [{escalus_new_assert,assert_true,2,
               [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
                {line,84}]},
     {mod_blocking_SUITE,'-remove_user_from_blocklist/1-fun-0-',2,
               [{file,"/home/circleci/project/big_tests/tests/mod_blocking_SUITE.erl"},
                {line,196}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1302}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1234}]}]}}

Report log

mod_blocking_SUITE:manage:add_many_users_to_blocklist
{error,
  {{badmatch,
     {error,
       {connection_step_failed,
         {{escalus_session,session},
          {client,
            <<"mike_unnamed_2622_add_many_users_to_blocklist_2627@localhost/res1">>,
            escalus_tcp,<0.64129.0>,undefined,
            [{event_client,
               [{event_manager,<0.64073.0>},
              {server,<<"localhost">>},
              {username,
                <<"mike_unnamed_2622_add_many_users_to_blocklist_2627">>},
              {resource,<<"res1">>}]},
             {resource,<<"res1">>},
             {username,
               <<"mike_unnamed_2622_add_many_users_to_blocklist_2627">>},
             {server,<<"localhost">>},
             {host,<<"localhost">>},
             {port,5222},
             {auth,{escalus_auth,auth_plain}},
             {wspath,undefined},
             {username,
               <<"mike_unnamed_2622_add_many_users_to_blocklist_2627">>},
             {server,<<"localhost">>},
             {password,<<"nicniema">>},
             {stream_id,<<"9a0f75c134b49997">>}]},
          [{compression,false},
           {starttls,true},
           {stream_management,true},
           {advanced_message_processing,true},
           {client_state_indication,false},
           {sasl_mechanisms,[<<"SCRAM-SHA-256">>,<<"PLAIN">>]},
           {caps,undefined}]},
         {timeout,session_reply}}}},
   [{escalus_story,'-start_ready_clients/2-fun-0-',3,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib...

Report log


internal_mnesia_26 / internal_mnesia / 915cbd3
Reports root/ big
OK: 2424 / Failed: 0 / User-skipped: 757 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 915cbd3
Reports root/ big
OK: 4930 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 915cbd3
Reports root/ big
OK: 4927 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0

Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

@chrzaszcz chrzaszcz merged commit e621614 into master Apr 16, 2024
4 checks passed
@chrzaszcz chrzaszcz deleted the ssl-tests-fix branch April 16, 2024 13:41
@jacekwegr jacekwegr added this to the 6.3.0 milestone Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants